-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: disable automatic eviction of diskcache at aggregation #419
fix: disable automatic eviction of diskcache at aggregation #419
Conversation
Codecov Report
❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more. @@ Coverage Diff @@
## main #419 +/- ##
=======================================
Coverage 14.86% 14.86%
=======================================
Files 48 48
Lines 2832 2832
=======================================
Hits 421 421
Misses 2382 2382
Partials 29 29 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
one question left; overall lgtm
self.cache = Cache() | ||
self.cache.reset('size_limit', 1e15) | ||
self.cache.reset('cull_limit', 0) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this the only place we have to change?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My bad. Applied changes to where it should be.
4548f06
to
7a4b1da
Compare
Python diskcache, which we use for aggregating trainer updates, have its own automatic eviction policy, depending on its size_limit and cull_limit value. Updated it to disable automatic eviction, and added a logger debug line that tells you the number of updates in the cache.
7a4b1da
to
83b1e52
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
Python diskcache, which we use for aggregating trainer updates, have its own automatic eviction policy, depending on its size_limit and cull_limit value. Updated it to disable automatic eviction, and added a logger debug line that tells you the number of updates in the cache.
Description
Please provide a meaningful description of what this change will do, or is for. Bonus points for including links to
related issues, other PRs, or technical references.
Note that by not including a description, you are asking reviewers to do extra work to understand the context of this
change, which may lead to your PR taking much longer to review, or result in it not being reviewed at all.
Type of Change
Checklist